{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dropout\n",
    "Dropout [1] is a technique for regularizing neural networks by randomly setting some features to zero during the forward pass. In this exercise you will implement a dropout layer and modify your fully-connected network to optionally use dropout.\n",
    "\n",
    "[1] Geoffrey E. Hinton et al, \"Improving neural networks by preventing co-adaptation of feature detectors\", arXiv 2012"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "run the following from the cs231n directory and try again:\n",
      "python setup.py build_ext --inplace\n",
      "You may also need to restart your iPython kernel\n"
     ]
    }
   ],
   "source": [
    "# As usual, a bit of setup\n",
    "from __future__ import print_function\n",
    "import time\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from cs231n.classifiers.fc_net import *\n",
    "from cs231n.data_utils import get_CIFAR10_data\n",
    "from cs231n.gradient_check import eval_numerical_gradient, eval_numerical_gradient_array\n",
    "from cs231n.solver import Solver\n",
    "\n",
    "%matplotlib inline\n",
    "plt.rcParams['figure.figsize'] = (10.0, 8.0) # set default size of plots\n",
    "plt.rcParams['image.interpolation'] = 'nearest'\n",
    "plt.rcParams['image.cmap'] = 'gray'\n",
    "\n",
    "# for auto-reloading external modules\n",
    "# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "def rel_error(x, y):\n",
    "  \"\"\" returns relative error \"\"\"\n",
    "  return np.max(np.abs(x - y) / (np.maximum(1e-8, np.abs(x) + np.abs(y))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_train:  (49000, 3, 32, 32)\n",
      "y_train:  (49000,)\n",
      "X_val:  (1000, 3, 32, 32)\n",
      "y_val:  (1000,)\n",
      "X_test:  (1000, 3, 32, 32)\n",
      "y_test:  (1000,)\n"
     ]
    }
   ],
   "source": [
    "# Load the (preprocessed) CIFAR10 data.\n",
    "\n",
    "data = get_CIFAR10_data()\n",
    "for k, v in data.items():\n",
    "  print('%s: ' % k, v.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dropout forward pass\n",
    "In the file `cs231n/layers.py`, implement the forward pass for dropout. Since dropout behaves differently during training and testing, make sure to implement the operation for both modes.\n",
    "\n",
    "Once you have done so, run the cell below to test your implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running tests with p =  0.3\n",
      "Mean of input:  10.0002078785\n",
      "Mean of train-time output:  6.99359371393\n",
      "Mean of test-time output:  10.0002078785\n",
      "Fraction of train-time output set to zero:  0.300672\n",
      "Fraction of test-time output set to zero:  0.0\n",
      "\n",
      "Running tests with p =  0.6\n",
      "Mean of input:  10.0002078785\n",
      "Mean of train-time output:  4.01406142322\n",
      "Mean of test-time output:  10.0002078785\n",
      "Fraction of train-time output set to zero:  0.598632\n",
      "Fraction of test-time output set to zero:  0.0\n",
      "\n",
      "Running tests with p =  0.75\n",
      "Mean of input:  10.0002078785\n",
      "Mean of train-time output:  2.50540643728\n",
      "Mean of test-time output:  10.0002078785\n",
      "Fraction of train-time output set to zero:  0.749504\n",
      "Fraction of test-time output set to zero:  0.0\n",
      "\n"
     ]
    }
   ],
   "source": [
    "np.random.seed(231)\n",
    "x = np.random.randn(500, 500) + 10\n",
    "\n",
    "for p in [0.3, 0.6, 0.75]:\n",
    "  out, _ = dropout_forward(x, {'mode': 'train', 'p': p})\n",
    "  out_test, _ = dropout_forward(x, {'mode': 'test', 'p': p})\n",
    "\n",
    "  print('Running tests with p = ', p)\n",
    "  print('Mean of input: ', x.mean())\n",
    "  print('Mean of train-time output: ', out.mean())\n",
    "  print('Mean of test-time output: ', out_test.mean())\n",
    "  print('Fraction of train-time output set to zero: ', (out == 0).mean())\n",
    "  print('Fraction of test-time output set to zero: ', (out_test == 0).mean())\n",
    "  print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dropout backward pass\n",
    "In the file `cs231n/layers.py`, implement the backward pass for dropout. After doing so, run the following cell to numerically gradient-check your implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dx relative error:  1.89289517961e-11\n"
     ]
    }
   ],
   "source": [
    "np.random.seed(231)\n",
    "x = np.random.randn(10, 10) + 10\n",
    "dout = np.random.randn(*x.shape)\n",
    "\n",
    "dropout_param = {'mode': 'train', 'p': 0.8, 'seed': 123}\n",
    "out, cache = dropout_forward(x, dropout_param)\n",
    "dx = dropout_backward(dout, cache)\n",
    "dx_num = eval_numerical_gradient_array(lambda xx: dropout_forward(xx, dropout_param)[0], x, dout)\n",
    "\n",
    "print('dx relative error: ', rel_error(dx, dx_num))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fully-connected nets with Dropout\n",
    "In the file `cs231n/classifiers/fc_net.py`, modify your implementation to use dropout. Specificially, if the constructor the the net receives a nonzero value for the `dropout` parameter, then the net should add dropout immediately after every ReLU nonlinearity. After doing so, run the following to numerically gradient-check your implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running check with dropout =  0\n",
      "Initial loss:  2.30305969832\n",
      "W1 relative error: 3.04e-07\n",
      "W2 relative error: 9.48e-06\n",
      "W3 relative error: 1.14e-07\n",
      "b1 relative error: 1.19e-08\n",
      "b2 relative error: 1.90e-08\n",
      "b3 relative error: 1.45e-10\n",
      "\n",
      "Running check with dropout =  0.25\n",
      "Initial loss:  2.3017488915\n",
      "W1 relative error: 1.86e-06\n",
      "W2 relative error: 1.54e-06\n",
      "W3 relative error: 8.30e-08\n",
      "b1 relative error: 2.21e-08\n",
      "b2 relative error: 1.91e-09\n",
      "b3 relative error: 2.36e-10\n",
      "\n",
      "Running check with dropout =  0.5\n",
      "Initial loss:  2.30278062267\n",
      "W1 relative error: 3.24e-06\n",
      "W2 relative error: 2.45e-06\n",
      "W3 relative error: 1.11e-07\n",
      "b1 relative error: 2.04e-07\n",
      "b2 relative error: 7.34e-08\n",
      "b3 relative error: 2.31e-10\n",
      "\n"
     ]
    }
   ],
   "source": [
    "np.random.seed(231)\n",
    "N, D, H1, H2, C = 2, 15, 20, 30, 10\n",
    "X = np.random.randn(N, D)\n",
    "y = np.random.randint(C, size=(N,))\n",
    "\n",
    "for dropout in [0, 0.25, 0.5]:\n",
    "  print('Running check with dropout = ', dropout)\n",
    "  model = FullyConnectedNet([H1, H2], input_dim=D, num_classes=C,\n",
    "                            weight_scale=5e-2, dtype=np.float64,\n",
    "                            dropout=dropout, seed=123)\n",
    "\n",
    "  loss, grads = model.loss(X, y)\n",
    "  print('Initial loss: ', loss)\n",
    "\n",
    "  for name in sorted(grads):\n",
    "    f = lambda _: model.loss(X, y)[0]\n",
    "    grad_num = eval_numerical_gradient(f, model.params[name], verbose=False, h=1e-5)\n",
    "    print('%s relative error: %.2e' % (name, rel_error(grad_num, grads[name])))\n",
    "  print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Regularization experiment\n",
    "As an experiment, we will train a pair of two-layer networks on 500 training examples: one will use no dropout, and one will use a dropout probability of 0.75. We will then visualize the training and validation accuracies of the two networks over time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "(Iteration 1 / 125) loss: 6.485370\n",
      "(Epoch 0 / 25) train acc: 0.194000; val_acc: 0.166000\n",
      "(Epoch 1 / 25) train acc: 0.282000; val_acc: 0.192000\n",
      "(Epoch 2 / 25) train acc: 0.262000; val_acc: 0.145000\n",
      "(Epoch 3 / 25) train acc: 0.226000; val_acc: 0.122000\n",
      "(Epoch 4 / 25) train acc: 0.200000; val_acc: 0.102000\n",
      "(Epoch 5 / 25) train acc: 0.190000; val_acc: 0.104000\n",
      "(Epoch 6 / 25) train acc: 0.196000; val_acc: 0.105000\n",
      "(Epoch 7 / 25) train acc: 0.200000; val_acc: 0.109000\n",
      "(Epoch 8 / 25) train acc: 0.206000; val_acc: 0.112000\n",
      "(Epoch 9 / 25) train acc: 0.200000; val_acc: 0.098000\n",
      "(Epoch 10 / 25) train acc: 0.200000; val_acc: 0.098000\n",
      "(Epoch 11 / 25) train acc: 0.202000; val_acc: 0.098000\n",
      "(Epoch 12 / 25) train acc: 0.202000; val_acc: 0.100000\n",
      "(Epoch 13 / 25) train acc: 0.204000; val_acc: 0.100000\n",
      "(Epoch 14 / 25) train acc: 0.206000; val_acc: 0.105000\n",
      "(Epoch 15 / 25) train acc: 0.220000; val_acc: 0.112000\n",
      "(Epoch 16 / 25) train acc: 0.220000; val_acc: 0.111000\n",
      "(Epoch 17 / 25) train acc: 0.224000; val_acc: 0.116000\n",
      "(Epoch 18 / 25) train acc: 0.226000; val_acc: 0.119000\n",
      "(Epoch 19 / 25) train acc: 0.224000; val_acc: 0.118000\n",
      "(Epoch 20 / 25) train acc: 0.232000; val_acc: 0.119000\n",
      "(Iteration 101 / 125) loss: 1.939167\n",
      "(Epoch 21 / 25) train acc: 0.230000; val_acc: 0.120000\n",
      "(Epoch 22 / 25) train acc: 0.230000; val_acc: 0.121000\n",
      "(Epoch 23 / 25) train acc: 0.230000; val_acc: 0.121000\n",
      "(Epoch 24 / 25) train acc: 0.228000; val_acc: 0.121000\n",
      "(Epoch 25 / 25) train acc: 0.228000; val_acc: 0.122000\n",
      "0.75\n",
      "(Iteration 1 / 125) loss: 2.872914\n",
      "(Epoch 0 / 25) train acc: 0.114000; val_acc: 0.114000\n",
      "(Epoch 1 / 25) train acc: 0.130000; val_acc: 0.104000\n",
      "(Epoch 2 / 25) train acc: 0.122000; val_acc: 0.098000\n",
      "(Epoch 3 / 25) train acc: 0.122000; val_acc: 0.092000\n",
      "(Epoch 4 / 25) train acc: 0.112000; val_acc: 0.092000\n",
      "(Epoch 5 / 25) train acc: 0.118000; val_acc: 0.089000\n",
      "(Epoch 6 / 25) train acc: 0.118000; val_acc: 0.089000\n",
      "(Epoch 7 / 25) train acc: 0.118000; val_acc: 0.089000\n",
      "(Epoch 8 / 25) train acc: 0.114000; val_acc: 0.087000\n",
      "(Epoch 9 / 25) train acc: 0.114000; val_acc: 0.087000\n",
      "(Epoch 10 / 25) train acc: 0.110000; val_acc: 0.087000\n",
      "(Epoch 11 / 25) train acc: 0.110000; val_acc: 0.087000\n",
      "(Epoch 12 / 25) train acc: 0.110000; val_acc: 0.087000\n",
      "(Epoch 13 / 25) train acc: 0.108000; val_acc: 0.087000\n",
      "(Epoch 14 / 25) train acc: 0.108000; val_acc: 0.087000\n",
      "(Epoch 15 / 25) train acc: 0.108000; val_acc: 0.087000\n",
      "(Epoch 16 / 25) train acc: 0.106000; val_acc: 0.087000\n",
      "(Epoch 17 / 25) train acc: 0.108000; val_acc: 0.087000\n",
      "(Epoch 18 / 25) train acc: 0.108000; val_acc: 0.087000\n",
      "(Epoch 19 / 25) train acc: 0.108000; val_acc: 0.087000\n",
      "(Epoch 20 / 25) train acc: 0.110000; val_acc: 0.087000\n",
      "(Iteration 101 / 125) loss: 2.290621\n",
      "(Epoch 21 / 25) train acc: 0.112000; val_acc: 0.087000\n",
      "(Epoch 22 / 25) train acc: 0.114000; val_acc: 0.087000\n",
      "(Epoch 23 / 25) train acc: 0.114000; val_acc: 0.087000\n",
      "(Epoch 24 / 25) train acc: 0.116000; val_acc: 0.088000\n",
      "(Epoch 25 / 25) train acc: 0.116000; val_acc: 0.088000\n"
     ]
    }
   ],
   "source": [
    "# Train two identical nets, one with dropout and one without\n",
    "np.random.seed(231)\n",
    "num_train = 500\n",
    "small_data = {\n",
    "  'X_train': data['X_train'][:num_train],\n",
    "  'y_train': data['y_train'][:num_train],\n",
    "  'X_val': data['X_val'],\n",
    "  'y_val': data['y_val'],\n",
    "}\n",
    "\n",
    "solvers = {}\n",
    "dropout_choices = [0, 0.75]\n",
    "for dropout in dropout_choices:\n",
    "  model = FullyConnectedNet([500], dropout=dropout)\n",
    "  print(dropout)\n",
    "\n",
    "  solver = Solver(model, small_data,\n",
    "                  num_epochs=25, batch_size=100,\n",
    "                  update_rule='adam',\n",
    "                  optim_config={\n",
    "                    'learning_rate': 5e-4,\n",
    "                  },\n",
    "                  verbose=True, print_every=100)\n",
    "  solver.train()\n",
    "  solvers[dropout] = solver"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4gAAAJNCAYAAABz4lt9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XmYXXWV8PvvopJAMYaQvAhJgGBiWiYTLYJIX5CoBBRD\nVIZgM4qN2tJ262teQg82l1dbGnwbL8rTV2xlUDQMjSFqa0QGvShDKiYmEoyEOQlKCCQyhJBh3T/O\nrnBSVCWnSJ3aqXO+n+c5T53928Nv7To7lVr1W/u3IzORJEmSJGmHsgOQJEmSJG0fTBAlSZIkSYAJ\noiRJkiSpYIIoSZIkSQJMECVJkiRJBRNESZIkSRJggihJEhHREhEvRsR+ZcciSVKZwucgSpL6m4h4\nsWpxZ2AtsKFY/kRm3tD3UUmS1P+ZIEqS+rWIeBz4eGb+fAvbDMjM9X0XVd9p5HOTJPU9S0wlSQ0n\nIr4YETdGxPcj4gXgjIg4MiLui4hVEfF0RFwZEQOL7QdEREbEAcXyd4v1P4mIFyLi3ogY1U1fO0TE\nLRHxx+LYd0fEW6vW7xwRV0TEkxGxOiJ+GRE7FuuOLmJaHRFPRcSZRfs9EXFO1TE+HhF3d4r1byJi\nCfD7ov3rEbE0Iv4cEXMi4l1V+w+IiH+OiEeK9e0RsW9EfCMi/q3T+fwkIi7Y9k9BktQfmSBKkhrV\nh4DvAXsANwLrgb8DhgJHAccDn9jC/h8F/hkYAjwJ/O8tbPsjYAzwJuB3wHeq1l0BHAYcURzrH4CN\nRcL5Y+Dfgb2A8cDCHpzfZOBw4NBi+f6inyHALcDNHYkoMA04mco5DwY+DrwCXAd8NCJ2AIiIvYFj\ngBk9iEOS1EBMECVJjeqezPxhZm7MzDWZOScz78/M9Zn5KHA1lWSoO7dkZntmrgNuAMZ1tVFx/Gsz\n84XMfAW4GHhHROwSES3AOcBnMvPpzNyQmfcUxzwD+Glm3lTE9Gxmzu/B+f1rZj6fmWuKOL6Tmc8V\n5aaXAbsDo4ttPw78Q2Y+XMQ7v9j218Caqu/D6cDPM/PZHsQhSWogJoiSpEb1VPVCRPxFRPy4KAX9\nM3AJldHE7vyx6v3LwK5dbVTMgHpZRDxaHHdJsWoosDcwCHiki11HdtNeq87n978i4vcRsRp4HtiF\n185vS31dTyVZpfj6nW62kyQ1ARNESVKj6jwL2zeolH+OzszdgS8A0Qv9nAW8H5hIpZy1Y9QugD8B\nrwJv7mK/p7ppB3iJyuysHd7UxTabzi8ijgU+B3yESgnpnsCLvHZ+W+rrO8CHI2J8sc0Pu9lOktQE\nTBAlSc1iN2A18FIxicyW7j/s6XHXAiupJHVf6liRmRuAa4GvRsSbitHGo4rJcb4LHB8RHykmkRka\nEW8rdp0PfCQiWiPiLcDHaohhPfAsMJBKmesuVev/E/hiRLw5KsZFxJAixieK/q4Dbi7KZCVJTcoE\nUZLULP4ncDbwApXRxBt76bjXAMuL14PArzut/yzwEDAXeA74VyqPmXoM+CBwYdH+G16bcOYrVEYI\nnwG+TSWZ3JL/Bn4OPAw8DvwZeLpq/eXATOCOYt3VwE5V668r+ra8VJKanM9BlCSpyUXEROBbwIHp\nLwaS1NQcQZQkqYlFxCAqj//4psmhJMkEUZKkJhURh1KZ8XQIcGXJ4UiStgOWmEqSJEmSAEcQJUmS\nJEkFE0RJkiRJEgADyg6gLwwdOjQPOOCAssOQJEmSpFLMnTv32cwctrXtmiJBPOCAA2hvby87DEmS\nJEkqRUQ8Uct2lphKkiRJkgATREmSJElSwQRRkiRJkgSYIEqSJEmSCiaIkiRJkiTABFGSJEmSVGiK\nx1yoYua8ZVw+ezHLV61h38GtTJs0linjh5cdliRJkqTthAlik5g5bxkX3bqQNes2ALBs1RouunUh\ngEmiJEmSJMAS06Zx+ezFm5LDDmvWbeDy2YtLikiSJEnS9sYEsUksX7WmR+2SJEmSmo8JYpPYd3Br\nj9olSZIkNR8TxCYxbdJYWge2bNbWOrCFaZPGlhSRJEmSpO2Nk9Q0iY6JaJzFVJIkSVJ3TBCbyJTx\nw00IJUmSJHWrriWmEXF8RCyOiCURMb2L9Z+LiEURsSAi7oiI/Yv2YyNiftXrlYiYUqy7NiIeq1o3\nrp7nIEmSJEnNom4jiBHRAlwFvA9YCsyJiFmZuahqs3lAW2a+HBGfAi4DTsvMu4BxxXGGAEuAn1Xt\nNy0zb6lX7JIkSZLUjOo5gjgBWJKZj2bmq8AM4KTqDTLzrsx8uVi8DxjRxXFOBn5StZ0kSZIkqQ7q\nmSAOB56qWl5atHXnPOAnXbRPBb7fqe1LRVnqFRGx47aFKUmSJEmC+iaI0UVbdrlhxBlAG3B5p/Z9\ngEOB2VXNFwF/ARwODAEu7OaY50dEe0S0r1ixoufRS5IkSVKTqWeCuBQYWbU8AljeeaOIeC/wj8Dk\nzFzbafWpwA8yc11HQ2Y+nRVrgWuolLK+TmZenZltmdk2bNiwbTwVSZIkSWp89UwQ5wBjImJURAyi\nUio6q3qDiBgPfINKcvhMF8c4nU7lpcWoIhERwBTgd3WIXZIkSZKaTt1mMc3M9RFxAZXy0Bbg25n5\nYERcArRn5iwqJaW7AjdX8j2ezMzJABFxAJURyF90OvQNETGMSgnrfOCT9ToHSZIkSWomkdnlbYEN\npa2tLdvb28sOQ5IkSZJKERFzM7Nta9vVs8RUkiRJktSPmCBKkiRJkgATREmSJElSwQRRkiRJkgSY\nIEqSJEmSCiaIkiRJkiTABFGSJEmSVDBBlCRJkiQBMKDsANQcZs5bxuWzF7N81Rr2HdzKtEljmTJ+\neNlhSZIkSapigqi6mzlvGRfdupA16zYAsGzVGi66dSGASaIkSZK0HbHEVHV3+ezFm5LDDmvWbeDy\n2YtLikiSJElSV0wQVXfLV63pUbskSZKkcpggqu72Hdzao3ZJkiRJ5TBBVN1NmzSW1oEtm7W1Dmxh\n2qSxJUUkSZIkqStOUqO665iIxllMJUmSpO2bCaL6xJTxw00IJUmSpO2cJaaSJEmSJMAEUZIkSZJU\nMEGUJEmSJAEmiJIkSZKkggmiJEmSJAkwQZQkSZIkFeqaIEbE8RGxOCKWRMT0LtZ/LiIWRcSCiLgj\nIvavWrchIuYXr1lV7aMi4v6IeDgiboyIQfU8B0mSJElqFnVLECOiBbgKOAE4CDg9Ig7qtNk8oC0z\nDwNuAS6rWrcmM8cVr8lV7f8GXJGZY4DngfPqdQ6SJEmS1EzqOYI4AViSmY9m5qvADOCk6g0y867M\nfLlYvA8YsaUDRkQAE6kkkwDXAVN6NWpJkiRJalL1TBCHA09VLS8t2rpzHvCTquWdIqI9Iu6LiI4k\ncC9gVWaur/GYkiRJkqQaDajjsaOLtuxyw4gzgDbgmKrm/TJzeUQcCNwZEQuBP/fgmOcD5wPst99+\nPYlbkiRJkppSPUcQlwIjq5ZHAMs7bxQR7wX+EZicmWs72jNzefH1UeBuYDzwLDA4IjoS2y6PWex3\ndWa2ZWbbsGHDtv1sJEmSJKnB1TNBnAOMKWYdHQRMBWZVbxAR44FvUEkOn6lq3zMidizeDwWOAhZl\nZgJ3AScXm54N3FbHc5AkSZKkplG3BLG4T/ACYDbwEHBTZj4YEZdERMespJcDuwI3d3qcxVuB9oj4\nLZWE8NLMXFSsuxD4XEQsoXJP4rfqdQ6SJEmS1EyiMijX2Nra2rK9vb3sMCRJkiSpFBExNzPbtrZd\nPUtMJUmSJEn9iAmiJEmSJAkwQZQkSZIkFUwQJUmSJEmACaIkSZIkqWCCKEmSJEkCTBAlSZIkSQUT\nREmSJEkSYIIoSZIkSSqYIEqSJEmSABNESZIkSVLBBFGSJEmSBJggSpIkSZIKJoiSJEmSJAAGlB2A\nJEmS1BdmzlvG5bMXs3zVGvYd3Mq0SWOZMn542WFJ2xUTREmSJDW8mfOWcdGtC1mzbgMAy1at4aJb\nFwKYJEpVLDGVJElSw7t89uJNyWGHNes2cPnsxSVFJG2fHEGUJElSw1u+ak2P2ntTWaWtZZbUWs7b\nf5kgSpIkqeHtO7iVZV0kg/sObq1rv2WVtpZZUms5b/9miakkSZIa3rRJY2kd2LJZW+vAFqZNGlvX\nfssqbS2zpLbMvmfOW8ZRl97JqOk/5qhL72TmvGV173N76Ls3OYIoSZKkhtcxctXXZY9llbaWWVJb\nVt+OmvYOE0Q1NOvfJUnqWjP+Hzll/PA+P8eySlvL6rfMvrc0clnvz73MvntbXUtMI+L4iFgcEUsi\nYnoX6z8XEYsiYkFE3BER+xft4yLi3oh4sFh3WtU+10bEYxExv3iNq+c5qP/q+EvOslVrSF77S05/\nHe6XJKm3+H9k3ymrtLWsfsvsuxlHTeuhbgliRLQAVwEnAAcBp0fEQZ02mwe0ZeZhwC3AZUX7y8BZ\nmXkwcDzw1YgYXLXftMwcV7zm1+sc1L85nbUkSV3z/8i+M2X8cL784UMZPriVAIYPbuXLHz607qNK\nZfVbZt/djVD21ahpWX33tnqWmE4AlmTmowARMQM4CVjUsUFm3lW1/X3AGUX7H6q2WR4RzwDDgFV1\njFcNppH+kiNJakxllXk24yMfylRGaWuZ/ZbV97RJYze7DxD6dtS0rL57Wz1LTIcDT1UtLy3aunMe\n8JPOjRExARgEPFLV/KWi9PSKiNixN4JV42mkv+RIkhpPmWWeZf0faWmr6qkZR03roZ4jiNFFW3a5\nYcQZQBtwTKf2fYDvAGdn5sai+SLgj1SSxquBC4FLujjm+cD5APvtt98bOwP1a430lxxJUuMpc1KL\nsv6PbKSJPLR9arZR03qo5wjiUmBk1fIIYHnnjSLivcA/ApMzc21V++7Aj4F/ysz7Otoz8+msWAtc\nQ6WU9XUy8+rMbMvMtmHDhvXKCal/aaS/5EiSGk+ZZZ5l/R/p7R/S9q+eI4hzgDERMQpYBkwFPlq9\nQUSMB74BHJ+Zz1S1DwJ+AFyfmTd32mefzHw6IgKYAvyujuegfq5R/pIjSWo8ZT6GAJrrkQ+Sale3\nEcTMXA9cAMwGHgJuyswHI+KSiJhcbHY5sCtwc/HIillF+6nA0cA5XTzO4oaIWAgsBIYCX6zXOUiS\npL4zc94yjrr0TkZN/zFHXXpnn96XVkbfZT6GoCzNeM5SfxOZXd4W2FDa2tqyvb297DAkSVI3OiYv\n6XxPXF+UPZbdd7PN6NmM5yxtDyJibma2bXU7E0RJklS2oy69s8vSw+GDW/nV9IkN27ck9ZVaE8R6\nTlIjSZJUkzInL3HiFEl6jQmiJEkqXZnPrvW5uZL0GhNESZJUujInL3HiFEl6TT0fcyFJklSTjklK\nypi8pMy+JWl74yQ1kiRJktTgnKRGkiRJktQjJoiSJEmSJMAEUZIkSZJUMEGUJEmSJAEmiJIkSZKk\nggmiJEmSJAmoIUGMiAsiYs++CEaSJEmSVJ5aRhDfBMyJiJsi4viIiHoHJUmSJEnqe1tNEDPzn4Ax\nwLeAc4CHI+JfI+LNdY5NkiRJktSHBtSyUWZmRPwR+COwHtgTuCUibs/M/1XPACVpezVz3jIun72Y\n5avWsO/gVqZNGsuU8cPLDkvaZl7bktS8tpogRsRngLOBZ4H/BKZl5rqI2AF4GDBBlNR0Zs5bxkW3\nLmTNug0ALFu1hotuXQjgL9Lq17y2Jam51XIP4lDgw5k5KTNvzsx1AJm5ETixrtFJ0nbq8tmLN/0C\n3WHNug1cPntxSRFJvcNrW5KaWy0lpv8NPNexEBG7AQdl5v2Z+VDdIpOk7djyVWt61C69EWWUenpt\nS1Jzq2UE8T+AF6uWXyraJKlp7Tu4tUftUk91lHouW7WG5LVSz5nzltW1X69tSWputSSIkZnZsVCU\nltY0uY0kNappk8bSOrBls7bWgS1MmzS2pIga38x5yzjq0jsZNf3HHHXpnXVPlMrut6xST69tSWpu\ntSR6jxYT1XSMGv4N8Gj9QpKk7V9HmZ8zPfaNsiZOKXPClrJKPb22Jam51ZIgfhK4EvgnIIE7gPNr\nOXhEHA/8P0AL8J+ZeWmn9Z8DPk7l0RkrgI9l5hPFurOLPgG+mJnXFe3vAK4FWqncH/l31SOcktRX\npowf7i/NfWRLo2n1/AzK6hcqJZ3LukgG+6LU02tbkprXVhPEzHwGmNrTA0dEC3AV8D5gKTAnImZl\n5qKqzeYBbZn5ckR8CrgMOC0ihgD/ArRRSUrnFvs+T2Uk83zgPioJ4vHAT3oan1RvPkes7zTj97qs\ncy6r37JG08qcsGXapLGbjV6CpZ6SpPqr5TmIOwHnAQcDO3W0Z+bHtrLrBGBJZj5aHGcGcBKwKUHM\nzLuqtr8POKN4Pwm4PTOfK/a9HTg+Iu4Gds/Me4v264EpmCBqO+NzxPpOM36vm7HcsqzRtLJH8cBS\nT0lS36plkprvAG+ikrT9AhgBvFDDfsOBp6qWlxZt3TmP1xK97vYdXryv9ZhSKXyOWN9pxu91Wedc\n5ve6rIlTyp6wZcr44fxq+kQeu/QD/Gr6RJNDSVLd1XIP4ujMPCUiTsrM6yLie8DsGvaLLtq6vFcw\nIs6gUk56zFb27ckxz6e4V3K//fbbWqxSr/I5Yn2nGb/XzVhuWdZomqN4kqRmU0uCuK74uioiDgH+\nCBxQw35LgZFVyyOA5Z03ioj3Av8IHJOZa6v2fXenfe8u2kds7ZgAmXk1cDVAW1ubk9ioT5VZltZs\nmvF73YzlllDexClO2CJJaia1lJheHRF7UplRdBaVewj/rYb95gBjImJURAyiMtHNrOoNImI88A1g\ncjEZTofZwHERsWfR93HA7Mx8GnghIt4ZEQGcBdxWQyxSnyq7LK0MZT0rrhm/181abilJkupviyOI\nEbED8Odi9tBfAgfWeuDMXB8RF1BJ9lqAb2fmgxFxCdCembOAy4FdgZsr+R5PZubkzHwuIv43lSQT\n4JKOCWuAT/HaYy5+ghPUaDvUbGVpZU5e0mzfa7DcUpIk1U9s7RGCEfHLzDy6j+Kpi7a2tmxvby87\njE2acUp+NbajLr2zy9LD4YNb+dX0iSVEJEmSpGoRMTcz27a2XS33IN4eEZ8HbgRe6misGtFTDzTj\nlPxqfM04UYwkSVIjquUexI8Bn6ZSYjq3eG0/w3H9TDNOya/G190kJY08UYwkSVIj2mqCmJmjunjV\nfC+iNudIixqRk5dIkiQ1hq2WmEbEWV21Z+b1vR9O4yt7mnipHpy8RJIkqTHUcg/i4VXvdwLeA/wG\nMEF8A6ZNGrvZPYjgSIt6V1mTIPmsOEmSpP5vqwliZv5t9XJE7AF8p24RNThHWlRPToIkSZKkbVHL\nCGJnLwNjejuQZuJIi+plS5Mgec1JkiRpa2q5B/GHQMfDEncADgJuqmdQkt4YJ0GSJEnStqhlBPEr\nVe/XA09k5tI6xSNpGzgJkiRJkrZFLc9BfBK4PzN/kZm/AlZGxAF1jUrSG+LjJiRJkrQtakkQbwY2\nVi1vKNokbWemjB/Olz98KMMHtxLA8MGtfPnDh3r/oSRJkmpSS4npgMx8tWMhM1+NiEF1jEnSNnAS\nJEmSJL1RtYwgroiIyR0LEXES8Gz9QpIkSZIklaGWEcRPAjdExNeL5aXAWfULSZIkSZJUhq0miJn5\nCPDOiNgViMx8of5hSZIkSZL62lZLTCPiXyNicGa+mJkvRMSeEfHFvghOkiRJktR3arkH8YTMXNWx\nkJnPA++vX0iSJEmSpDLUkiC2RMSOHQsR0QrsuIXtJUmSJEn9UC2T1HwXuCMirimWzwWuq19IkiRJ\nkqQy1DJJzWURsQB4LxDAT4H96x2YJEmSJKlv1VJiCvBHYCPwEeA9wEN1i0iSJEmSVIpuRxAj4i3A\nVOB0YCVwI5XHXBzbR7FJkiRJkvrQlkYQf09ltPCDmfmXmfk1YENPDh4Rx0fE4ohYEhHTu1h/dET8\nJiLWR8TJVe3HRsT8qtcrETGlWHdtRDxWtW5cT2KSJEmSJHVtS/cgfoTKCOJdEfFTYAaVexBrEhEt\nwFXA+4ClwJyImJWZi6o2exI4B/h89b6ZeRcwrjjOEGAJ8LOqTaZl5i21xiJJkiRJ2rpuRxAz8weZ\neRrwF8DdwGeBvSPiPyLiuBqOPQFYkpmPZuarVBLMkzr18XhmLqByf2N3TgZ+kpkv19CnJEmSJOkN\n2uokNZn5UmbekJknAiOA+cDrykW7MBx4qmp5adHWU1OB73dq+1JELIiIK6qf0ShJkiRJeuNqncUU\ngMx8LjO/kZkTa9i8q3LU7El/EbEPcCgwu6r5IiqjmocDQ4ALu9n3/Ihoj4j2FStW9KRbSZIkSWpK\nPUoQe2gpMLJqeQSwvIfHOBX4QWau62jIzKezYi1wDZVS1tfJzKszsy0z24YNG9bDbiVJkiSp+dQz\nQZwDjImIURExiEqp6KweHuN0OpWXFqOKREQAU4Df9UKskiRJktT06pYgZuZ64AIq5aEPATdl5oMR\ncUlETAaIiMMjYilwCvCNiHiwY/+IOIDKCOQvOh36hohYCCwEhgJfrNc5SJIkSVIzicwe3RbYL7W1\ntWV7e3vZYUiSJElSKSJibma2bW27epaYSpIkSZL6ERNESZIkSRJggihJkiRJKpggSpIkSZIAE0RJ\nkiRJUsEEUZIkSZIEmCBKkiRJkgomiJIkSZIkwARRkiRJklQwQZQkSZIkASaIkiRJkqSCCaIkSZIk\nCTBBlCRJkiQVTBAlSZIkSYAJoiRJkiSpYIIoSZIkSQJMECVJkiRJBRNESZIkSRJggihJkiRJKpgg\nSpIkSZIAE0RJkiRJUsEEUZIkSZIE1DlBjIjjI2JxRCyJiOldrD86In4TEesj4uRO6zZExPziNauq\nfVRE3B8RD0fEjRExqJ7nIEmSJEnNom4JYkS0AFcBJwAHAadHxEGdNnsSOAf4XheHWJOZ44rX5Kr2\nfwOuyMwxwPPAeb0evCRJkiQ1oXqOIE4AlmTmo5n5KjADOKl6g8x8PDMXABtrOWBEBDARuKVoug6Y\n0nshS5IkSVLzqmeCOBx4qmp5adFWq50ioj0i7ouIjiRwL2BVZq5/g8eUJEmSJHVjQB2PHV20ZQ/2\n3y8zl0fEgcCdEbEQ+HOtx4yI84HzAfbbb78edCtJkiRJzameI4hLgZFVyyOA5bXunJnLi6+PAncD\n44FngcER0ZHYdnvMzLw6M9sys23YsGE9j16SJEmSmkw9E8Q5wJhi1tFBwFRg1lb2ASAi9oyIHYv3\nQ4GjgEWZmcBdQMeMp2cDt/V65JIkSZLUhOqWIBb3CV4AzAYeAm7KzAcj4pKImAwQEYdHxFLgFOAb\nEfFgsftbgfaI+C2VhPDSzFxUrLsQ+FxELKFyT+K36nUOkiRJktRMojIo19ja2tqyvb297DAkSZIk\nqRQRMTcz27a2XT1LTCVJkiRJ/YgJoiRJkiQJMEGUJEmSJBVMECVJkiRJgAmiJEmSJKlggihJkiRJ\nAkwQJUmSJEkFE0RJkiRJEmCCKEmSJEkqmCBKkiRJkgATREmSJElSwQRRkiRJkgSYIEqSJEmSCiaI\nkiRJkiTABFGSJEmSVDBBlCRJkiQBJojNZcFNcMUhcPHgytcFN5UdkSRJkqTtyICyA1AfWXAT/PAz\nsG5NZXn1U5VlgMNOLS8uSZIkSdsNRxCbxR2XvJYcdli3ptIuSZIkSZggNo/VS3vWLkmSJKnpmCA2\niz1G9KxdkiRJUtMxQWwW7/kCDGzdvG1ga6VdkiRJkqhzghgRx0fE4ohYEhHTu1h/dET8JiLWR8TJ\nVe3jIuLeiHgwIhZExGlV666NiMciYn7xGlfPc2gYh50KH7wS9hgJROXrB690ghpJkiRJm9RtFtOI\naAGuAt4HLAXmRMSszFxUtdmTwDnA5zvt/jJwVmY+HBH7AnMjYnZmrirWT8vMW+oVe8M67FQTQkmS\nJEndqudjLiYASzLzUYCImAGcBGxKEDPz8WLdxuodM/MPVe+XR8QzwDBgFZIkSZKkuqhnielw4Kmq\n5aVFW49ExARgEPBIVfOXitLTKyJix20LU5IkSZIE9U0Qo4u27NEBIvYBvgOcm5kdo4wXAX8BHA4M\nAS7sZt/zI6I9ItpXrFjRk25VDwtugisOgYsHV74uuKmx+5UkSZL6oXomiEuBkVXLI4Dlte4cEbsD\nPwb+KTPv62jPzKezYi1wDZVS1tfJzKszsy0z24YNG/aGTkC9ZMFN8MPPwOqngKx8/eFn6p+sldWv\nJEmS1E/VM0GcA4yJiFERMQiYCsyqZcdi+x8A12fmzZ3W7VN8DWAK8LtejVq9745LYN2azdvWram0\nN2K/kiRJUj9VtwQxM9cDFwCzgYeAmzLzwYi4JCImA0TE4RGxFDgF+EZEPFjsfipwNHBOF4+zuCEi\nFgILgaHAF+t1Duolq5f2rL2/9ytJkiT1U/WcxZTM/G/gvzu1faHq/Rwqpaed9/su8N1ujjmxl8NU\nve0xoijz7KK9EfuVJEmS+ql6lphKFe/5Agxs3bxtYGulvRH7LVszTgjkZESSJEm9oq4jiBIAh51a\n+XrHJZXyzj1GVJK0jvZG67dMHRPzdNx72TExD9T3vMvqt+y+JUmSGkxk9ujJE/1SW1tbtre3lx2G\nVH9XHNJNWe1I+Gwd53Mqq9+y+5YkSeonImJuZrZtbTtLTKVG0owTAjkZkSRJ6uAtL9vMBFFqJN1N\nwNMXEwKV0W/ZfUuSpO1Hmc/AbqDnb5sglqFB/rqg7VAzTghUZt/+lVKStD1rtv+nynwGdgM9f9t7\nEPta5wk1oPLL7AevdEIN9Y4FN5UzMU9Z/ZbVd5n/lv05IqnBrFu3jqVLl/LKK6+UHUrjePUlWPMc\nVP+uHwGtQ2DQLo3Z96onu183eL/69Vt2353stNNOjBgxgoEDB27WXus9iCaIfc0JNaTG4MQ8ktRr\nHnvsMXbbbTf22msvIqLscBrDnx6EDa++vr1lEOx9cGP23Yzn3ElmsnLlSl544QVGjRq12Tonqdle\nOaFG87AEsLE5MY8k9ZpXXnmlsZPDl5+rJBDL51W+vvxc/fvsKlnZUnsj9L3bPhCd0pvYodJeb2X2\nXd1lBHvmcg7HAAAgAElEQVTttdc2jcabIPY1J9RoDg10o7K64cQ8ktSrGjo5XP3Ua8nRhlcry/VO\nElsG9ay9EfreeUilmqajn5ZBleWdh9S337L77mRb/y2ZIPa1MifUUN9poBuV1Q0n5mmOCQ/K7FdS\n79vCSN5Pf/pTxo4dy+jRo7n00ku73H3t2rWcdtppjB49miOOOILHH39807ovf/nLjB49mrFjxzJ7\n9uzXdnrhaciNmx8oN3LtN/+DCy64oDfPbnNbGdH66le/yssvv1xK3/Ww6fM7bAKXXvND2Hd8pbSz\nKkH77Gc/y7hx4xg3bhxvectbGDx48KZ1LS0tm9ZNnjx5q/09/vjjHHLIIZs37jyk0mcXffe2mTNn\nsmjRoroce0BdjqrudUwgUdZkHuoblgA2vjL/LZfVd+fJcTpGxqtjarS+yzxnSb2rYySvI1nrGMkD\nNuy4B5/+9Ke5/fbbGTFiBIcffjiTJ0/moIMO2uwQ3/rWt9hzzz1ZsmQJM2bM4MILL+TGG29k0aJF\nzJgxgwcffJDly5fz3ve+lz/84Q+0tLR0X1a5cX2XzevXr2fAgF74Fb0jOXnh6UoMLYMqCVrR/tWv\nfpUzzjiDnXfeedv76mHfvW3Dhg01fX5XXHHFpvdf+9rXmDdv3qbl1tZW5s+fv82x9NrntxUzZ87k\nxBNPfN059gZHEMtw2KmViSQuXlX56i8ZjccSwOZQ5r/lMvpuxunDrQaQtksz5y3jqEvvZNT0H3PU\npXcyc96yre/UzUgeLzzNAw88wOjRoznwwAMZNGgQU6dO5bbbbnvdIW677TbOPvtsAE4++WTuuOMO\nMpPbbruNqVOnsuOOOzJq1ChGjx7NAw88UNmpqqzymhtv4y1/OYVjPvJxfjV34ab2c845h8997nMc\ne+yxXHjhhTz33HNMmTKFww47jHe+850sWLAAgIsvvpgzzzyTiRMnMmbMGL75zW9WTiOTadOmccgh\nh3DooYdy4403AnD3Aws48bwLN41oXfC/vsC1117LlVdeyfLlyzn22GM59thja/6+98iWRtN6uTKj\n1s+v2ve//31OP/30HvUzd+5c3va2t3HkkUdy1VVXbWq/9tprOeWUU/jgBz/Icccd1/3ncffdHH30\n0XzoQx/ioIMO4pOf/CQbN27cFM+hhx7KIYccwoUXXrjp2Lvuuuum97fccgvnnHMOv/71r5k1axbT\npk1j3LhxPPLIIz06j61xBFGqh/d8oevHEFhKrP6sGSfmKbsaoNkeW1Pm43LK0myPCOqu34GH1rz7\nzHnLuOjWhaxZtwGAZavWcNGtlWRryvjh3e+4hYlTli1bxsiRIzc1jRgxgvvvv/91m1ZvN2DAAPbY\nYw9WrlzJsmXLeOc737nZ/suWFUnrbvvA6qd4+o9/4l++8g3m/vQG9th9d46d+mnGv+NNm/b5wx/+\nwM9//nNaWlr427/9W8aPH8/MmTO58847OeusszaNbi1YsID77ruPl156ifHjx/OBD3yAe++9l/nz\n5/Pb3/6WZ599lsMPP5yjjz6622/FZz7zGf793/+du+66i6FDh3b/PauHOlRm1Pr5dXjiiSd47LHH\nmDhx4qa2V155hba2NgYMGMD06dOZMmXK6/Y799xz+drXvsYxxxzDtGnTNlt37733smDBAoYMGcJ/\n/dd/dft5PPDAAyxatIj999+f448/nltvvZV3vetdXHjhhcydO5c999yT4447jpkzZ3YZA8C73vUu\nJk+ezIknnsjJJ5/co+9VLRxBlOrhsFMrz6TbYyQQla8+o079XTNOzFPmOZc12VWz9VumMs95e/uc\nX32p5kNcPnvxpuSww5p1G7h89uIt77iFiVO6euxbVxN9dLfdFvcvJi+5/7e/591HvoNh/2NvBg07\nkNNO/6vNtj/llFMqJanAPffcw5lnngnAxIkTWblyJatXrwbgpJNOorW1laFDh3LsscfywAMPcM89\n93D66afT0tLC3nvvzTHHHMOcOXO6/16UqQ6VGbV+fh1mzJjBySefvOn7DfDkk0/S3t7O9773Pf7+\n7//+daNyq1evZtWqVRxzzDEAmz6fDu973/sYMqQySrqlz2PChAkceOCBtLS0cPrpp3PPPfcwZ84c\n3v3udzNs2DAGDBjAX/3VX/HLX/7yjX0zeoEJolQvlhKr0TTjxDxlnnOzldWWXc5bxmRElm2/1u8r\nq2s+xPJVa3rUvskWJk4ZMWIETz312vNlly5dyr777vu6Q1Rvt379elavXs2QIUO2vv/OQ2DwfsQu\ne3U7eckuu7z2APktJTydE5/uElSojHJ2lDAC2/Tog15Th8qMWj+/DjNmzHhdeWnH9gceeCDvfve7\nN7s/ESqfyZaSzq19fh168vl13r6vPj8TRElSbcocGS+r7zLPudnKasss5y1rNM2y7dd0M2FLV/Yd\n3Nqj9k228BiCww8/nIcffpjHHnuMV199lRkzZnQ5k+XkyZO57rrrgMr9YBMnTiQimDx5MjNmzGDt\n2rU89thjPPzww0yYMGGzfY844gjuvvtuVq5cybp167j55pu7DfXoo4/mhhtuACr3rQ0dOpTdd98d\nqNwH+corr7By5UruvvvuTeWLN954Ixs2bGDFihX88pe/ZMKECey///4sWrSItWvXsnr1au64445N\nfey222688MILW/6e1UMdKjNq/fwAFi9ezPPPP8+RRx65qe35559n7dq1ADz77LP86le/et3kL4MH\nD2aPPfbgnnvuAdj0+XSlu88DKiWmjz32GBs3buTGG2/kL//yLzniiCP4xS9+wbPPPsuGDRv4/ve/\nv2mkcu+99+ahhx5i48aN/OAHP9jURz0/P+9BlCTV7rBTyxsNL6vvsvrdY8SmGRZf126/vWtLo2n1\n/OzLPOft7XPeofZfSadNGrvZPYgArQNbmDZp7NZ33nlIl6N3AwYM4Otf/zqTJk1iw4YNfOxjH+Pg\ngw8G4Atf+AJtbW1MnjyZ8847jzPPPJPRo0czZMgQZsyYAcDBBx/MqaeeykEHHcSAAQO46qqrNitf\nBNhnn324+OKLOfLII9lnn314+9vfzoYNG14XC1Qmozn33HM57LDD2HnnnTclpVApUfzABz7Ak08+\nyT//8z+z77778qEPfYh7772Xt73tbUQEl112GW96U+X+xlNPPZXDDjuMMWPGMH78+E3HOf/88znh\nhBPYZ599uOuuu7b+vestdZinodbPDyqTwUydOnWzkbmHHnqIT3ziE+ywww5s3LiR6dOndzk76DXX\nXMPHPvYxdt55ZyZNmtRtPN19Hr///e858sgjmT59OgsXLtw0Yc0OO+zAl7/8ZY499lgyk/e///2c\ndNJJAFx66aWceOKJjBw5kkMOOYQXX3wRgKlTp/LXf/3XXHnlldxyyy28+c1vfsPfv85iS0OajaKt\nrS3b29vLDkOSpNp1nsgBKr9E1XsEs9n6hUpZKV39PhSV2wTqpcxz3s4+54dO/BFvfVtbzYeZOW8Z\nl89ezPJVa9h3cCvTJo3d8gQ1DeLiiy9m11135fOf/3zZoWybZpyQispo8Fe+8hV+9KMf1b2vhx56\niLe+9a2btUXE3Mzc6j80RxAlSdoelfW8y2brF8obTWvG56l21+/AXba8XydTxg9vioSwYZVZjaKt\ncgRRkiQ1tzJH8gR0Pdoh6Y3blhFEJ6mRJEnNzUcTSdImdU0QI+L4iFgcEUsiYnoX64+OiN9ExPqI\nOLnTurMj4uHidXZV+zsiYmFxzCtjS/PNSpIk1cJHE5WuGarapL6wrf+W6pYgRkQLcBVwAnAQcHpE\ndJ4O6EngHOB7nfYdAvwLcAQwAfiXiNizWP0fwPnAmOJ1fJ1OQZIkSX1gp512YuXKlSaJ0jbKTFau\nXMlOO+30ho9Rz0lqJgBLMvNRgIiYAZwELOrYIDMfL9Zt7LTvJOD2zHyuWH87cHxE3A3snpn3Fu3X\nA1OAn9TxPCRJklRHI0aMYOnSpaxYsaLsUKR+b6eddmLEiDc+yVY9E8ThQPWUYEupjAi+0X2HF6+l\nXbRLkiSpnxo4cCCjRo0qOwxJ1PcexK7uDay1bqC7fWs+ZkScHxHtEdHuX6MkSZIkaevqmSAuBUZW\nLY8Alm/jvkuL91s9ZmZenZltmdk2bNiwmoOWJEmSpGZVzwRxDjAmIkZFxCBgKjCrxn1nA8dFxJ7F\n5DTHAbMz82nghYh4ZzF76VnAbfUIXpIkSZKaTdRztqiIeD/wVaAF+HZmfikiLgHaM3NWRBwO/ADY\nE3gF+GNmHlzs+zHgH4pDfSkzryna24BrgVYqk9P8bW7lJCJiBfBEb59fLxgKPFt2EGpYXl+qJ68v\n1ZPXl+rNa0z1tL1eX/tn5lZLK+uaIGrLIqI9M9vKjkONyetL9eT1pXry+lK9eY2pnvr79VXPElNJ\nkiRJUj9igihJkiRJAkwQy3Z12QGooXl9qZ68vlRPXl+qN68x1VO/vr68B1GSJEmSBDiCKEmSJEkq\nmCCWICKOj4jFEbEkIqaXHY8aT0Q8HhELI2J+RLSXHY/6t4j4dkQ8ExG/q2obEhG3R8TDxdc9y4xR\n/Vc319fFEbGs+Bk2v3hsltRjETEyIu6KiIci4sGI+Lui3Z9h2mZbuL769c8wS0z7WES0AH8A3gcs\nBeYAp2fmolIDU0OJiMeBtszcHp/Bo34mIo4GXgSuz8xDirbLgOcy89LiD117ZuaFZcap/qmb6+ti\n4MXM/EqZsan/i4h9gH0y8zcRsRswF5gCnIM/w7SNtnB9nUo//hnmCGLfmwAsycxHM/NVYAZwUskx\nSVK3MvOXwHOdmk8CriveX0flP0Spx7q5vqRekZlPZ+ZvivcvAA8Bw/FnmHrBFq6vfs0Ese8NB56q\nWl5KA1xI2u4k8LOImBsR55cdjBrS3pn5NFT+gwT+R8nxqPFcEBELihJUy/+0zSLiAGA8cD/+DFMv\n63R9QT/+GWaC2PeiizbrfNXbjsrMtwMnAJ8uSrgkqb/4D+DNwDjgaeD/lBuO+ruI2BX4L+DvM/PP\nZcejxtLF9dWvf4aZIPa9pcDIquURwPKSYlGDyszlxddngB9QKW2WetOfinsvOu7BeKbkeNRAMvNP\nmbkhMzcC38SfYdoGETGQyi/vN2TmrUWzP8PUK7q6vvr7zzATxL43BxgTEaMiYhAwFZhVckxqIBGx\nS3GjNBGxC3Ac8Lst7yX12Czg7OL92cBtJcaiBtPxi3vhQ/gzTG9QRATwLeChzPz3qlX+DNM26+76\n6u8/w5zFtATFVLdfBVqAb2fml0oOSQ0kIg6kMmoIMAD4nteYtkVEfB94NzAU+BPwL8BM4CZgP+BJ\n4JTMdKIR9Vg319e7qZRmJfA48ImO+8WknoiIvwT+P2AhsLFo/gcq94n5M0zbZAvX1+n0459hJoiS\nJEmSJMASU0mSJElSwQRRkiRJkgSYIEqSJEmSCiaIkiRJkiTABFGSJEmSVDBBlCSpRhGxISLmV72m\n9+KxD4iIfvWsLElS4xlQdgCSJPUjazJzXNlBSJJUL44gSpK0jSLi8Yj4t4h4oHiNLtr3j4g7ImJB\n8XW/on3viPhBRPy2eL2rOFRLRHwzIh6MiJ9FRGtpJyVJakomiJIk1a61U4npaVXr/pyZE4CvA18t\n2r4OXJ+ZhwE3AFcW7VcCv8jMtwFvBx4s2scAV2XmwcAq4CN1Ph9JkjYTmVl2DJIk9QsR8WJm7tpF\n++PAxMx8NCIGAn/MzL0i4llgn8xcV7Q/nZlDI2IFMCIz11Yd4wDg9swcUyxfCAzMzC/W/8wkSapw\nBFGSpN6R3bzvbpuurK16vwHnCpAk9TETREmSesdpVV/vLd7/GphavP8r4J7i/R3ApwAioiUidu+r\nICVJ2hL/MilJUu1aI2J+1fJPM7PjURc7RsT9VP74enrR9hng2xExDVgBnFu0/x1wdUScR2Wk8FPA\n03WPXpKkrfAeREmStlFxD2JbZj5bdiySJG0LS0wlSZIkSYAjiJIkSZKkgiOIkiQVIuKAiMiI8B59\nSVJTMkGUJDWMiJgdEZd00X5SRPzRxE+SpC0zQZQkNZJrgTMjIjq1nwnckJnr+z6k3hEV/r8tSaor\n/6ORJDWSmcAQ4P/qaIiIPYETgeuL5Q9ExLyI+HNEPBURF9d68IiYHhGPRMQLEbEoIj7Uaf1fR8RD\nVevfXrSPjIhbI2JFRKyMiK8X7RdHxHer9t+sxDUi7o6IL0XEr4CXgQMj4tyqPh6NiE90iuGkiJhf\nnN8jEXF8RJwSEXM7bfc/I2JmrecuSWoOJoiSpIaRmWuAm4CzqppPBX6fmb8tll8q1g8GPgB8KiKm\n1NjFI1SSzz2A/xv4bkTsAxARpwAXF8feHZgMrIyIFuBHwBPAAcBwYEYPTutM4Hxgt+IYz1BJeHen\n8lzFK6oS0QlUEuFpxfkdDTwOzAJGRcRbq457BvCdHsQhSWoCJoiSpEZzHXBKRLQWy2cVbQBk5t2Z\nuTAzN2bmAuD7wDG1HDgzb87M5cW+NwIPAxOK1R8HLsvMOVmxJDOfKNbvC0zLzJcy85XMvKcH53Nt\nZj6Ymeszc11m/jgzHyn6+AXwM14bMT0P+HZm3l7EuCwzf5+Za4EbqSSFRMTBVJLVH/UgDklSEzBB\nlCQ1lCL5WgGcFBEHAocD3+tYHxFHRMRdRbnnauCTwNBajh0RZxXlm6siYhVwSNW+I6mMMHY2Enhi\nG+5/fKpTDCdExH0R8VwRw/triAEqSfJHi/szzwRuKhJHSZI2MUGUJDWi66mMHJ4J/Cwz/1S17ntU\nSi5HZuYewP8LdJ7U5nUiYn/gm8AFwF6ZORj4XdW+TwFv7mLXp4D9uplB9SVg56rlN3WxzaYHFkfE\njsB/AV8B9i5i+O8aYiAz7wNepTLa+FEsL5UkdcEEUZLUiK4H3gv8NVXlpYXdgOcy85Xinr2P1njM\nXagkaysAIuJcKiOIHf4T+HxEvKOYcXR0kVQ+ADwNXBoRu0TEThFxVLHPfODoiNgvIvYALtpKDIOA\nHYsY1kfECcBxVeu/BZwbEe+JiB0iYnhE/EXV+uuBrwPre1jmKklqEiaIkqSGk5mPA7+mktTN6rT6\nb4BLIuIF4AtUJrWp5ZiLgP8D3Av8CTgU+FXV+puBL1EZoXyBYkbVzNwAfBAYDTwJLAVOK/a5ncq9\ngQuAuWzlnsDMfAH4TBHz81SS21lV6x+gmLgGWA38Ati/6hDfoZLUOnooSepSZObWt5IkSf1eMXHP\nM8DbM/PhsuORJG1/HEGUJKl5fAqYY3IoSepOVzfMS5KkBhMRj1OZzKbWZz5KkpqQJaaSJEmSJMAS\nU0mSJElSwQRRkiRJkgQ0yT2IQ4cOzQMOOKDsMCRJkiSpFHPnzn02M4dtbbumSBAPOOAA2tvbyw5D\nkiRJkkoREU/Usp0lppIkSZIkwARRkiRJklQwQZQkSZIkASaIkiRJkqSCCaIkSZIkCTBBlCRJkiQV\nmuIxF6qYOW8Zl89ezPJVa9h3cCvTJo1lyvjhZYclSZIkaTthgtgkZs5bxkW3LmTNug0ALFu1hotu\nXQhgkihJkiQJsMS0aVw+e/Gm5LDDmnUbuHz24pIikiRJkrS9MUFsEstXrelRuyRJkqTmY4LYJPYd\n3NqjdkmSJEnNxwSxSUybNJbWgS2btbUObGHapLElRSRJkiRpe+MkNU2iYyIaZzGVJEmS1B0TxCYy\nZfxwE0JJkiRJ3bLEVJIkSZIEmCBKkiRJkgomiJIkSZIkwARRkiRJklQwQZQkSZIkASaIkiRJkqSC\nCaIkSZIkCTBBlCRJkiQVTBAlSZIkSYAJoiRJkiSpYIIoSZIkSQJMECVJkiRJBRNESZIkSRJggihJ\nkiRJKpggSpIkSZIAE0RJkiRJUqGUBDEijo+IxRGxJCKmd7H+6Ij4TUSsj4iTO627LCIejIiHIuLK\niIi+i1ySJEmSGlefJ4gR0QJcBZwAHAScHhEHddrsSeAc4Hud9n0XcBRwGHAIcDhwTJ1DliRJkqSm\nMKCEPicASzLzUYCImAGcBCzq2CAzHy/Wbey0bwI7AYOAAAYCf6p/yJIkSZLU+MooMR0OPFW1vLRo\n26rMvBe4C3i6eM3OzId6PUJJkiRJakJlJIhd3TOYNe0YMRp4KzCCSlI5MSKO7mbb8yOiPSLaV6xY\n8YaDlSRJkqRmUUaCuBQYWbU8Alhe474fAu7LzBcz80XgJ8A7u9owM6/OzLbMbBs2bNg2BSxJkiRJ\nzaCMBHEOMCYiRkXEIGAqMKvGfZ8EjomIARExkMoENZaYSpIkSVIv6PMEMTPXAxcAs6kkdzdl5oMR\ncUlETAaIiMPj/2/v7qOkKu8Ej39/aSB01NC+DRFaIwYlIoK4DUrMomAiOlFEowjjJJLNOU5m5DjJ\nrKy4myjLzEYmxMQ18eTETBLNbCIwWUSc5IQYX+Jg3EAbEAYJvhLtbicqComxUV6e/aOqm6ZpoLur\nqy5V9f2c06fqPnXvfX636lZ1/ep5uRFNwJXAtyNiQ37zHwPPA+uBp4CnUkoPlPoYJEmSJKkSRUrd\nGv5X1hoaGlJjY2PWYUiSJElSJiLiyZRSw8HWy6KLqSRJkiTpEGSCKEmSJEkCTBAlSZIkSXkmiJIk\nSZIkwARRkiRJkpRngihJkiRJAkwQJUmSJEl5JoiSJEmSJMAEUZIkSZKUZ4IoSZIkSQJMECVJkiRJ\neSaIkiRJkiTABFGSJEmSlGeCKEmSJEkCTBAlSZIkSXkmiJIkSZIkwARRkiRJkpRngihJkiRJAkwQ\nJUmSJEl5JoiSJEmSJMAEUZIkSZKU1y/rAKrRsjXNLFyxiZatrQypq2XOlBFMGzs067AkSZIkVTkT\nxBJbtqaZm5aup3XHLgCat7Zy09L1ACaJkiRJkjJlF9MSW7hiU3ty2KZ1xy4WrtiUUUSSJEmSlGOC\nWGItW1t7VC5JkiRJpWKCWGJD6mp7VC5JkiRJpWKCWGJzpoygtn/NXmW1/WuYM2VERhFJkiRJUo6T\n1JRY20Q0zmIqSZIk6VBjgpiBaWOHmhBKkiRJOuTYxVSSJEmSBJggSpIkSZLyTBAlSZIkSUBGCWJE\nXBgRmyLiuYiY28XjEyPiNxGxMyKu6PTYCRHx84jYGBFPR8SJpYpbkiRJkipZyRPEiKgB7gQuAkYC\nMyNiZKfVXgJmAT/qYhc/ABamlE4FxgOvFi9aSZIkSaoeWcxiOh54LqX0AkBELAIuBZ5uWyGltDn/\n2O6OG+YTyX4ppQfz671VopglSZIkqeJl0cV0KPByh+WmfFl3nAJsjYilEbEmIhbmWyQlSZIkSQXK\nIkGMLspSN7ftB/xn4AZgHHASua6o+1YScW1ENEZE42uvvdabOCVJkiSpqmSRIDYBx3dYrgdaerDt\nmpTSCymlncAy4MyuVkwp3ZVSakgpNRx77LEFBSxJkiRJ1SCLBHE1cHJEDIuIAcAMYHkPtj0yItoy\nvsl0GLsoSZIkSeq9kieI+Za/2cAKYCOwJKW0ISLmR8RUgIgYFxFNwJXAtyNiQ37bXeS6lz4UEevJ\ndVf9TqmPQZIkSZIqUaTU3eF/5auhoSE1NjZmHYYkSZIkZSIinkwpNRxsvSy6mEqSJEmSDkEmiJIk\nSZIkwARRkiRJkpRngihJkiRJAkwQJUmSJEl5JoiSJEmSJMAEUZIkSZKUZ4IoSZIkSQJMECVJkiRJ\neSaIkiRJkiTABFGSJEmSlGeCKEmSJEkCTBAlSZIkSXkmiJIkSZIkwARRkiRJkpRngihJkiRJAkwQ\nJUmSJEl5BSWIETE7Io7sq2AkSZIkSdkptAXxA8DqiFgSERdGRPRFUJIkSZKk0isoQUwpfRE4Gfgu\nMAt4NiK+HBEf6oPYJEmSJEklVPAYxJRSAv4j/7cTOBL4cUR8pdB9S5IkSZJKp18hG0fE9cA1wOvA\nPwFzUko7IuI9wLPAfys8REmSJElSKRSUIALHAJenlH7XsTCltDsiLi5w35IkSZKkEio0Qfwp8Ebb\nQkQcAYxMKf06pbSxwH2rgixb08zCFZto2drKkLpa5kwZwbSxQ7MOS5IkSVIHhY5B/BbwVoflP+XL\npHbL1jRz09L1NG9tJQHNW1u5ael6lq1pzjo0SZIkSR0UmiBGfpIaINe1lMJbJVVhFq7YROuOXXuV\nte7YxcIVmzKKSJIkSVJXCk0QX4iI6yOif/7vb4EX+iIwVY6Wra09KpckSZKUjUITxM8BHwGagSbg\nLODaQoNSZRlSV9ujckmSJEnZKChBTCm9mlKakVL6s5TS4JTSX6SUXu2r4FQZ5kwZQW3/mr3KavvX\nMGfKiIwikiRJktSVQq+DOBD4LHAaMLCtPKX0XwqMSxWkbbZSZzGVJEmSDm2FTijzz8BvgSnAfOBq\nwMtbaB/Txg41IZQkSZIOcYWOQRyeUvoS8KeU0j3AJ4DTCw9LkiRJklRqhSaIO/K3WyNiFDAIOLHA\nfUqSJEmSMlBognhXRBwJfBFYDjwN/OPBNoqICyNiU0Q8FxFzu3h8YkT8JiJ2RsQVXTz+/ohojohv\nFhi/JEmSJCmv12MQI+I9wB9SSm8CjwEndXO7GuBO4OPkLo2xOiKWp5Se7rDaS8As4Ib97ObvgV/2\nMnRJkiRJUhd63YKYUtoNzO7FpuOB51JKL6SU3gUWAZd22vfmlNI6YHfnjSPiPwGDgZ/3om5JkiRJ\n0n4U2sX0wYi4ISKOj4ij2v4Oss1Q4OUOy035soPKt1reBszpXbiSJEmSpP0p9DIXbdc7vK5DWeLA\n3U2ji7LUzfr+BvhpSunliK5206GSiGuBawFOOOGEbu5ekiRJkqpXQQliSmlYLzZrAo7vsFwPtHRz\n2wnAf46IvwEOBwZExFsppX0mukkp3QXcBdDQ0NDdBFSSJEmSqlZBCWJEfLqr8pTSDw6w2Wrg5IgY\nBjQDM4C/6E59KaWrO9Q9C2joKjmUJEmSJPVcoV1Mx3W4PxA4H/gNsN8EMaW0MyJmAyuAGuB7KaUN\nEWoA7NYAABzpSURBVDEfaEwpLY+IccB9wJHAJRHxP1NKpxUYqyRJkiTpACKlvut9GRGDgH9OKU3t\ns532gYaGhtTY2Jh1GJIkSZKUiYh4MqXUcLD1Cp3FtLO3gZP7eJ+SJEmSpBIodAziA+yZgfQ9wEhg\nSaFBSZIkSZJKr9AxiF/tcH8n8LuUUlOB+5QkSZIkZaDQBPEl4JWU0naAiKiNiBNTSpsLjkySJEmS\nVFKFjkH8F2B3h+Vd+TJJkiRJUpkpNEHsl1J6t20hf39AgfuUJEmSJGWg0ATxtYhov6RFRFwKvF7g\nPiVJkiRJGSh0DOLngB9GxDfzy03ApwvcpyRJkiQpAwUliCml54GzI+JwIFJKf+ybsCRJkiRJpVZQ\nF9OI+HJE1KWU3kop/TEijoyIf+ir4CRJkiRJpVPoGMSLUkpb2xZSSm8Cf17gPiVJkiRJGSg0QayJ\niPe2LURELfDeA6wvSZIkSTpEFTpJzf8BHoqI7+eXPwPcU+A+JUmSJEkZKHSSmq9ExDrgY0AAPwM+\n2BeBSZIkSZJKq9AupgD/AewGPgmcD2zsg31KkiRJkkqsVy2IEXEKMAOYCWwBFpO7zMWkPoxNkiRJ\nklRCve1i+lvg34BLUkrPAUTEF/osKkmSJElSyfW2i+knyXUtfSQivhMR55MbgyhJkiRJKlO9ShBT\nSvellK4CPgw8CnwBGBwR34qIC/owPkmSJElSiRQ0SU1K6U8ppR+mlC4G6oG1wNw+iUySJEmSVFJ9\nMYspACmlN1JK304pTe6rfUqSJEmSSqfPEkRJkiRJUnkzQZQkSZIkASaIkiRJkqQ8E0RJkiRJEmCC\nKEmSJEnKM0GUJEmSJAEmiJIkSZKkPBNESZIkSRIA/bIOQCqmZWuaWbhiEy1bWxlSV8ucKSOYNnZo\n1mFJkiSpwlTK904TRFWsZWuauWnpelp37AKgeWsrNy1dD1CWb1ZJkiQdmirpe6cJoirWwhWb2t+k\nbVp37GLhik1l90aVJEnqqSxbtLKqO6t6K+l7ZyZjECPiwojYFBHPRcTcLh6fGBG/iYidEXFFh/Iz\nIuKJiNgQEesi4qrSRq5y0rK1tUflkiRJlaKtRat5ayuJPS1ay9Y0V2zdWR5zJX3vLHmCGBE1wJ3A\nRcBIYGZEjOy02kvALOBHncrfBj6dUjoNuBC4PSLqihuxytWQutoelUuSJBXDsjXNnLPgYYbN/Qnn\nLHi4JAnLgVq0KrXuLI+5kr53ZtGCOB54LqX0QkrpXWARcGnHFVJKm1NK64DdncqfSSk9m7/fArwK\nHFuasFVu5kwZQW3/mr3KavvXMGfKiIwikiRJ1SarVq0sW7SyqjvLY66k751ZjEEcCrzcYbkJOKun\nO4mI8cAA4Pk+iksVpq2/dyXMJiVJUl+rlBkXeyKLY85qbNqQulqau0iMStGilVXdWR5zJX3vzCJB\njC7KUo92EHEc8M/ANSml3ftZ51rgWoATTjihpzGqQkwbO7Qs35iSJBVTJc242F1ZHXNWrVpzpozY\n63ihdC1aWdWd5TFD5XzvzKKLaRNwfIfleqCluxtHxPuBnwBfTCn9v/2tl1K6K6XUkFJqOPZYe6FK\nkiS1yXKsVlayOuasxqZNGzuUWy8/naF1tQQwtK6WWy8/vSQJTFZ1Z3nMlSSLFsTVwMkRMQxoBmYA\nf9GdDSNiAHAf8IOU0r8UL0RJkqTKleVYray6tlZjS16WLVpZ1V0prXhZKnkLYkppJzAbWAFsBJak\nlDZExPyImAoQEeMiogm4Evh2RGzIbz4dmAjMioi1+b8zSn0MkiRJ5SyrVq0sL0NQjS15Um9ESj0a\n/leWGhoaUmNjY9ZhSJIkHRI6j8eDXKtWsROXcxY83OUkIkPranl87uSi1QvZHbN0qIiIJ1NKDQdb\nL4suppIkScpQVjMuZtm1tZJmmZSKyQRRkiQpI1leaiKLsVpZXoYAHJ8mdUcWs5hKkiRVvSzH42Wl\nki4mLlUqE0RJkqQMVOOlJpywRTr02cVUkiQpA1mOx8uS3TylQ5stiFlYtwS+Pgrm1eVu1y3JOiJJ\nklRiWV12QZIOxASx1NYtgQeuh20vAyl3+8D1JomSJFUZx+NJOhSZIJbaQ/NhR6euIztac+WSJKlq\nOB5P0qHIMYiltq2pZ+WSJKliOR5P0qHGBLHUBtXnu5d2US6prGR5/TKpEvmekqTs2cW01M6/Gfp3\nGnzevzZXLqlsVOP1y6Ri8j0lSYcGE8RSGz0dLrkDBh0PRO72kjty5ZLKRjVev0wqpqzfU8vWNHPO\ngocZNvcnnLPgYRNTSVXLLqZZGD3dhFAqc9V6/TKpWLJ8T7W1XrYlqG2tl4BdXCVVHVsQJakXvH6Z\n1LeyfE9l3XopSYcSE0RJ6gWvXyb1rSzfU/YIkKQ97GIqqexlMfNh2/6dcVHqG1m+p4bU1dLcRTJo\njwBJ1ShSSlnHUHQNDQ2psbEx6zAkFUHnsUOQa3XwYtOSusvPEUnVICKeTCk1HGw9u5hKKmuOHZJU\nqGljh3Lr5acztK6WAIbW1ZocSqpadjGVVNYcOySpL0wbO9SEUJKwBVFSmXM2UUmSpL5jgiiprDmb\nqCRJUt+xi6mksuZsopIkSX3HBFFS2XPskCRJUt+wi6kkSZIkCbAFUVIfyeJi9ZKKw/ezJFUvE0RJ\nBet8kenmra3ctHQ9gF8qpTLj+1mSqptdTKvJuiXw9VEwry53u25J1hGpCJataeacBQ8zbO5POGfB\nwyxb01z0Or1YvVQ5fD9LUnWzBbFarFsCD1wPO/IXD9/2cm4ZYPT07OJSn8rql38vVi9VDt/PklTd\nbEGsFg/N35McttnRmitXxcjql38vVi9VDt/PklTdTBCrxbamnpWrLGX1y78Xq5cqh+9nSapudjGt\nFoPqc91KuypXUWQxC+CQulqau0gGi/3LvxerlyqH72dJqm6RUso6hqJraGhIjY2NWYeRrc5jEAH6\n18IldzgGsQg6jwWE3C/wt15+elG/ZGVVryRJkg5tEfFkSqnhYOvZxbRajJ6eSwYHHQ9E7tbksGiy\nGgs4bexQbr38dIbW1RLA0Lpak0NJkiR1WyZdTCPiQuB/AzXAP6WUFnR6fCJwOzAamJFS+nGHx64B\nvphf/IeU0j2liboCjJ5uQlgiWc4COG3sUBNCSZIk9UrJE8SIqAHuBD4ONAGrI2J5SunpDqu9BMwC\nbui07VHALUADkIAn89u+WYrYpe7KaiygpL6XxXjiQ6FuSVJ1yqKL6XjguZTSCymld4FFwKUdV0gp\nbU4prQN2d9p2CvBgSumNfFL4IHBhKYKWesJZAKXK0Daut3lrK4k91xZdtqa5ouuWJFWvLBLEoUDH\n6TSb8mXF3lYqGccCSpUhq/HEWdctSapeWYxBjC7KujuVare3jYhrgWsBTjjhhG7uXuo7jgWUyl+W\n44mzrFuSVL2ySBCbgOM7LNcDLT3Y9rxO2z7a1YoppbuAuyB3mYueBqk+tm4JPDQftjXlrr14/s1O\nmCMVIKuxadU2Hi/L8cSOZZYkZSGLLqargZMjYlhEDABmAMu7ue0K4IKIODIijgQuyJfpUNZ2DcZt\nLwMpd/vA9blyST2W1di0ahyPl+V4YscyS5KyUPIEMaW0E5hNLrHbCCxJKW2IiPkRMRUgIsZFRBNw\nJfDtiNiQ3/YN4O/JJZmrgfn5Mh3KHpoPOzr9Cr6jNVcuqceyGptWjePxshxP7FhmSVIWMrkOYkrp\np8BPO5Xd3OH+anLdR7va9nvA94oaoPrWtqaelUs6oKzGplXreLwsxxM7llmSVGpZdDFVtRnUZa6/\n/3JJB7S/MWjFHpuWVb1Z1y1JUjUxQVTxnX8z9O/0Ja5/ba5cUo9lNTbN8XiSJFW+TLqYqsq0zVbq\nLKZSn2jrcljqGT2zqjfruiVJqiaRUuVfAaKhoSE1NjZmHYYkSZIkZSIinkwpNRxsPbuYSpIkSZIA\nE0RJkiRJUp4JoiRJkiQJMEGUJEmSJOWZIEqSJEmSABNEVbp1S+Dro2BeXe523ZLqqFuSJEnqBa+D\nqMq1bgk8cD3saM0tb3s5twzFvwZjlnVLklRmduzYQVNTE9u3b886FKnsDRw4kPr6evr379+r7U0Q\nVbkemr8nQWuzozVXXuwkLcu6JUkqM01NTRxxxBGceOKJRETW4UhlK6XEli1baGpqYtiwYb3ah11M\nVbm2NfWsvFLqliSpzGzfvp2jjz7a5FAqUERw9NFHF9Qab4KoyjWovmfllVK3JEllyORQ6huFvpdM\nEFW5zr8Z+tfuXda/NldeyXVLkqQe+9nPfsaIESMYPnw4CxYs6HKdd955h6uuuorhw4dz1llnsXnz\n5vbHbr31VoYPH86IESNYsWLFQeu7++67mT17dl+F32O33347b7/9dmb197XuvH5f+MIXOOOMMzjj\njDM45ZRTqKura3+spqam/bGpU6cetL7NmzczatSoPou/p5YtW8bTTz9dlH07BlGVq22s30Pzc107\nB9XnErRSjAHMsu51S6qr3qzrliSV3LI1zSxcsYmWra0MqatlzpQRTBs7tNf727VrF9dddx0PPvgg\n9fX1jBs3jqlTpzJy5Mi91vvud7/LkUceyXPPPceiRYu48cYbWbx4MU8//TSLFi1iw4YNtLS08LGP\nfYxnnnmGmpqaHseyc+dO+vUr/lf022+/nb/8y7/kfe97X9Hr2kcf/9/u7uv39a9/vf3+N77xDdas\nWdO+XFtby9q1a3sdQ5tSvX7Lli3j4osv3ucY+4ItiKpso6fDF/4d5m3N3ZYyacii7rbZU7e9DKQ9\ns6cW+xIbWdWbdd2SpJJbtqaZm5aup3lrKwlo3trKTUvXs2xNc6/3uWrVKoYPH85JJ53EgAEDmDFj\nBvfff/8+691///1cc801AFxxxRU89NBDpJS4//77mTFjBu9973sZNmwYw4cPZ9WqVfts//3vf59T\nTjmFc889l8cff7y9fNasWfzd3/0dkyZN4sYbb+SNN95g2rRpjB49mrPPPpt169YBMG/ePD71qU8x\nefJkTj75ZL7zne8AuYlJ5syZw6hRozj99NNZvHgxAI8++igXX3xxez2zZ8/m7rvv5o477qClpYVJ\nkyYxadKkXj9vvVKE/9vdff06uvfee5k5c2aP6nnyyScZM2YMEyZM4M4772wvv/vuu7nyyiu55JJL\nuOCCCw74ekycOJHLLruMkSNH8rnPfY7du3e3x3P66aczatQobrzxxvZ9H3744e33f/zjHzNr1ix+\n9atfsXz5cubMmcMZZ5zB888/36PjOBgTRKmSHGj21EqsN+u6JUklt3DFJlp37NqrrHXHLhau2NTr\nfTY3N3P88ce3L9fX19PcvG/C2XG9fv36MWjQILZs2dKt7V955RVuueUWHn/8cR588MF9ugc+88wz\n/OIXv+C2227jlltuYezYsaxbt44vf/nLfPrTn25fb926dfzkJz/hiSeeYP78+bS0tLB06VLWrl3L\nU089xS9+8QvmzJnDK6+8st/jvf766xkyZAiPPPIIjzzySM+erEIV4f/2Xs//229QfwQ0P7MWfr8B\n3n5jn/V/97vf8eKLLzJ58uT2su3bt9PQ0MDZZ5/NsmXLuqznM5/5DHfccQdPPPHEPo898cQT3POt\n23n43m+w9J9uY+2qlTz1xCP7vB6rVq3itttuY/369Tz//PMsXbqUlpYWbrzxRh5++GHWrl3L6tWr\n9xsDwEc+8hGmTp3KwoULWbt2LR/60Id68nQdlAmiVEmymj3VGWMlSSXSsrW1R+XdkVLap6yriT72\nt153tv/1r3/Neeedx7HHHsuAAQO46qqr9nr8yiuvbO+SunLlSj71qU8BMHnyZLZs2cK2bdsAuPTS\nS6mtreWYY45h0qRJrFq1ipUrVzJz5kxqamoYPHgw5557LqtXr+7m0ZdYEf5vtz//b7+Ra5HcvTP3\n/O96N7fcKUlctGgRV1xxxV5dgF966SUaGxv50Y9+xOc///l9WuW2bdvG1q1bOffccwHaX582H580\nkaNq3oJd77Jy1RpmXnoBNW+1MPiI/nu9HuPHj+ekk06ipqaGmTNnsnLlSlavXt1+bvTr14+rr76a\nxx57rNfPR6FMEKVKktXsqdU6Y+y6JfD1UTCvLndbym6tWdXtMXvMlVhvlnV7zD2ud0hdbY/K9/L2\nG7lWpZY1e7Uu1dfX8/LLL7ev1tTUxJAhQ/bZvON6O3fuZNu2bRx11FEH3/7tN2DrS0Trm/tt1Trs\nsMPa7x8o4eyceO4vQQXot/Ntdrf+of14t7+1rcv1imI/z3Ux/m+3P/9/fAXSbppeeZUhg4/NPZh2\n58o7WLRo0T7dS9ter5NOOonzzjtvr/GJkHtNDjQ76GH983XR4fXrou6evH6d1y/k0hU9YYIoVZKs\nZk+txhljq3HcpcfsMVdivVnW7THvqffdP3V7F3OmjKC2/96Tv9T2r2HOlBEH3rCtdWnXu7nlDq1L\n48aN49lnn+XFF1/k3XffZdGiRV3OZDl16lTuueceIDcebPLkyUQEU6dOZdGiRbzzzju8+OKLPPvs\ns4wfP36ves8a82EefaKRLa+9yo7XX+BfFt+731AnTpzID3/4QyA3bu2YY47h/e9/P5AbB7l9+3a2\nbNnCo48+yrhx45g4cSKLFy9m165dvPbaazz22GOMH3UyH6yr4elnnuOdd95l25tbeOjhh+GdtwA4\n4ogj+OMf/3jQ57tXDvBcF+P/9t6v3w4W3b+CqRecu2eFtjiATZs28eabbzJhwoT2sjfffJN33nkH\ngNdff53HH398n8lf6urqGDRoECtXrgRof33apT3dnieefSaLl/8893q8+vvc65E/H1atWsWLL77I\n7t27Wbx4MR/96Ec566yz+OUvf8nrr7/Orl27uPfee9tbKgcPHszGjRvZvXs39913X3sdxXz9TBCl\nSjJ6OlxyBww6Hojc7SV3FH+CnKzqzbLuahx36TGXrt4s6662erOs22PeU+/27rdsTRs7lFsvP52h\ndbUEMLSullsvP/3gs5jmW5f2km/h6devH9/85jeZMmUKp556KtOnT+e0004D4Oabb2b58uUAfPaz\nn2XLli0MHz6cr33ta+2XUzjttNOYPn06I0eO5MILL+TOO+/c030xX+9xg49l3n/9KyZMncXHrvor\nzhy5/3Fj8+bNo7GxkdGjRzN37tz2pBRyXRQ/8YlPcPbZZ/OlL32JIUOGcNlllzF69GjGjBnD5MmT\n+cpXvsIHDtvN8UP+jOmXfJzRH7uKq2f/D8aOGtH+XF977bVcdNFFxZmk5gDPdTH+b7e/flfP5tTz\nPsn0Sz7OaSNyz+/NC7/F8l/smRDo3nvvZcaMGXu1zG3cuJGGhgbGjBnDpEmTmDt3bpezg37/+9/n\nuuuuY8KECdTWdkpyY8+PFpddNJnRp57MmI/PYPL0z+Vejw98AIAJEyYwd+5cRo0axbBhw7jssss4\n7rjjuPXWW5k0aRJjxozhzDPP5NJLLwVgwYIFXHzxxUyePJnjjjuuvY4ZM2awcOFCxo4d2+eT1MSB\nmjQrRUNDQ2psbMw6DEmVZF4d0NXnZ+Rmrq3Euj3m0tWbZd3VVm+WdXvM7TZOWcKpE6YUr17IdXXc\nnyFjy6LeefPmcfjhh3PDDTeUtN4ey6rutpbLjslpvCeXhL7vqOLV2826H330Ub761a/yr//6r8WN\nhVzSe+qpp+5VFhFPppQaDratLYiS1BvVOO7SYy5dvVnWXW31Zlm3x7zHe0pwae6aAT0rt97yq/t9\nR+USsrZ6agaUJjnMuu4+ZoIoSb1RjeMuPebS1Ztl3dVWb5Z1e8x76h04qLj1AhxxXK5Fp6N4T668\nTOqdN29e91oP+7jeHsuy7vcdBYNPy7VUDj6ttAnaQeo+77zzStJ6WCgTREnqjWocd+kxe8yVWG+W\ndXvMe+odcNhBNy1YVi081VZv1nWrYI5BlCRJUqY2btzIhz/84QNeRkBS96SU+O1vf+sYREmSJJWn\ngQMHsmXLlgNeD07SwaWU2LJlCwMHDuz1PkowIliSJEnav/r6epqamnjttdeyDkUqewMHDqS+vvcT\nTpkgSpIkKVP9+/dn2LBhWYchCbuYSpIkSZLyTBAlSZIkSYAJoiRJkiQpryoucxERrwG/yzqOLhwD\nvJ51EKpYnl8qJs8vFZPnl4rNc0zFdKieXx9MKR17sJWqIkE8VEVEY3euRSL1hueXisnzS8Xk+aVi\n8xxTMZX7+WUXU0mSJEkSYIIoSZIkScozQczWXVkHoIrm+aVi8vxSMXl+qdg8x1RMZX1+OQZRkiRJ\nkgTYgihJkiRJyjNBzEBEXBgRmyLiuYiYm3U8qjwRsTki1kfE2ohozDoelbeI+F5EvBoR/96h7KiI\neDAins3fHplljCpf+zm/5kVEc/4zbG1E/HmWMap8RcTxEfFIRGyMiA0R8bf5cj/DVLADnF9l/Rlm\nF9MSi4ga4Bng40ATsBqYmVJ6OtPAVFEiYjPQkFI6FK/BozITEROBt4AfpJRG5cu+AryRUlqQ/6Hr\nyJTSjVnGqfK0n/NrHvBWSumrWcam8hcRxwHHpZR+ExFHAE8C04BZ+BmmAh3g/JpOGX+G2YJYeuOB\n51JKL6SU3gUWAZdmHJMk7VdK6THgjU7FlwL35O/fQ+4fotRj+zm/pD6RUnolpfSb/P0/AhuBofgZ\npj5wgPOrrJkglt5Q4OUOy01UwImkQ04Cfh4RT0bEtVkHo4o0OKX0CuT+QQJ/lnE8qjyzI2Jdvguq\n3f9UsIg4ERgL/Bo/w9THOp1fUMafYSaIpRddlNnPV33tnJTSmcBFwHX5LlySVC6+BXwIOAN4Bbgt\n23BU7iLicOD/Ap9PKf0h63hUWbo4v8r6M8wEsfSagOM7LNcDLRnFogqVUmrJ374K3Eeua7PUl36f\nH3vRNgbj1YzjUQVJKf0+pbQrpbQb+A5+hqkAEdGf3Jf3H6aUluaL/QxTn+jq/Cr3zzATxNJbDZwc\nEcMiYgAwA1iecUyqIBFxWH6gNBFxGHAB8O8H3krqseXANfn71wD3ZxiLKkzbF/e8y/AzTL0UEQF8\nF9iYUvpah4f8DFPB9nd+lftnmLOYZiA/1e3tQA3wvZTS/8o4JFWQiDiJXKshQD/gR55jKkRE3Auc\nBxwD/B64BVgGLAFOAF4CrkwpOdGIemw/59d55LpmJWAz8Fdt48WknoiIjwL/BqwHdueL/zu5cWJ+\nhqkgBzi/ZlLGn2EmiJIkSZIkwC6mkiRJkqQ8E0RJkiRJEmCCKEmSJEnKM0GUJEmSJAEmiJIkSZKk\nPBNESZK6KSJ2RcTaDn9z+3DfJ0ZEWV0rS5JUefplHYAkSWWkNaV0RtZBSJJULLYgSpJUoIjYHBH/\nGBGr8n/D8+UfjIiHImJd/vaEfPngiLgvIp7K/30kv6uaiPhORGyIiJ9HRG1mByVJqkomiJIkdV9t\npy6mV3V47A8ppfHAN4Hb82XfBH6QUhoN/BC4I19+B/DLlNIY4ExgQ778ZODOlNJpwFbgk0U+HkmS\n9hIppaxjkCSpLETEWymlw7so3wxMTim9EBH9gf9IKR0dEa8Dx6WUduTLX0kpHRMRrwH1KaV3Ouzj\nRODBlNLJ+eUbgf4ppX8o/pFJkpRjC6IkSX0j7ef+/tbpyjsd7u/CuQIkSSVmgihJUt+4qsPtE/n7\nvwJm5O9fDazM338I+GuAiKiJiPeXKkhJkg7EXyYlSeq+2ohY22H5ZymltktdvDcifk3ux9eZ+bLr\nge9FxBzgNeAz+fK/Be6KiM+Sayn8a+CVokcvSdJBOAZRkqQC5ccgNqSUXs86FkmSCmEXU0mSJEkS\nYAuiJEmSJCnPFkRJkiRJEmCCKEmSJEnKM0GUJEmSJAEmiJIkSZKkPBNESZIkSRJggihJkiRJyvv/\nv1gijygbZTAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fe8f0911ba8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot train and validation accuracies of the two models\n",
    "\n",
    "train_accs = []\n",
    "val_accs = []\n",
    "for dropout in dropout_choices:\n",
    "  solver = solvers[dropout]\n",
    "  train_accs.append(solver.train_acc_history[-1])\n",
    "  val_accs.append(solver.val_acc_history[-1])\n",
    "\n",
    "plt.subplot(3, 1, 1)\n",
    "for dropout in dropout_choices:\n",
    "  plt.plot(solvers[dropout].train_acc_history, 'o', label='%.2f dropout' % dropout)\n",
    "plt.title('Train accuracy')\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend(ncol=2, loc='lower right')\n",
    "  \n",
    "plt.subplot(3, 1, 2)\n",
    "for dropout in dropout_choices:\n",
    "  plt.plot(solvers[dropout].val_acc_history, 'o', label='%.2f dropout' % dropout)\n",
    "plt.title('Val accuracy')\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend(ncol=2, loc='lower right')\n",
    "\n",
    "plt.gcf().set_size_inches(15, 15)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Question\n",
    "Explain what you see in this experiment. What does it suggest about dropout?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Answer\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
